Remote access SSH

Secure Shell (SSH) is the standard method for encrypted remote administration on Linux servers. In production environments, SSH is most commonly secured using key-based authentication, where a private key on the client proves identity to the server and the server authorizes access by matching that identity against ~/.ssh/authorized_keys.

Key-based SSH is widely adopted because it:

• Eliminates password brute-force as a primary entry path (when password auth is disabled)
• Supports automation and repeatable access patterns (CI/CD, maintenance scripts, config management)
• Works well with hardening controls such as rate limiting, IP allow-lists, jump hosts, and short-lived credentials

Scope

Focus: connecting to a remote server when the key pair already exists and the server already has your public key installed.

Assumptions

• The private key stays on the local machine and must never be shared.
• The public key is already present on the server under the target user account.
• You know the remote username, host/IP, and SSH port.

Concepts and Terminology

Term	Meaning
SSH client	The program used to connect (example: ssh)
SSH server (sshd)	The daemon running on the remote server (sshd)
Private key	Secret credential stored locally (example: ~/.ssh/id_ed25519)
Public key	Non-secret credential installed on the server (authorized_keys)
Host key	Server identity key used to prevent man-in-the-middle attacks (stored in known_hosts)
Passphrase	Optional encryption for your private key (recommended)
SSH agent	Holds decrypted keys in memory to avoid repeated passphrase prompts

Authentication Flow (Key-Based)

1. Client opens a connection to the server on the SSH port.
2. Server presents its host key; client verifies it against known_hosts (or prompts on first connect).
3. Client offers a public key identity.
4. Server checks whether that public key is allowed for the target user (authorized_keys).
5. Client proves it owns the matching private key (cryptographic challenge/response).
6. Server grants a session if checks pass.

Local and Remote File Layout

Local machine (client)

Common key names (OpenSSH):

• ~/.ssh/id_ed25519 (recommended)
• ~/.ssh/id_rsa (legacy)
• Custom key: ~/.ssh/mykey

Files:

File	Purpose
--	-
~/.ssh/<key>	Private key (secret)
~/.ssh/<key>.pub	Public key (shareable)
~/.ssh/config	Optional connection aliases and settings
~/.ssh/known_hosts	Saved server host keys (trust store)

Remote server

Files:

File	Purpose
--	-
~/.ssh/authorized_keys	List of allowed public keys for the user
/etc/ssh/sshd_config	SSH daemon configuration
/var/log/auth.log (Ubuntu/Debian)	Authentication logs

Permissions (Critical)

SSH refuses to use keys if permissions are too open.

Recommended permissions

Target	Path	Permission
		-
Local .ssh directory	~/.ssh	700
Local private key	~/.ssh/mykey	600
Local public key	~/.ssh/mykey.pub	644
Local SSH config	~/.ssh/config	600
Remote .ssh directory	~/.ssh	700
Remote authorized_keys	~/.ssh/authorized_keys	600

Fix permissions (local)

chmod 700 ~/.ssh
chmod 600 ~/.ssh/mykey
chmod 644 ~/.ssh/mykey.pub
chmod 600 ~/.ssh/config 2>/dev/null || true

Fix permissions (remote)

Run on the server as the target user:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Confirm ownership (remote)

ls -ld ~/.ssh
ls -l ~/.ssh/authorized_keys

Server-Side SSH Daemon Settings

Key lines in /etc/ssh/sshd_config:

PubkeyAuthentication yes

Hardening lines (apply after confirming key login works):

PasswordAuthentication no
PermitRootLogin no

Apply changes:

sudo systemctl restart ssh

Validate SSH configuration before restarting

sudo sshd -t

No output means the config syntax is valid.

Connecting from Linux, macOS, or WSL

Standard connection (default port 22)

ssh -i ~/.ssh/mykey user@server_ip

Custom port (example: 2581)

ssh -i ~/.ssh/mykey -p 2581 user@server_ip

Enforce key usage (avoid wrong key selection)

ssh -o IdentitiesOnly=yes -i ~/.ssh/mykey -p 2581 user@server_ip

Verbose debug mode (recommended when troubleshooting)

ssh -vvv -o IdentitiesOnly=yes -i ~/.ssh/mykey -p 2581 user@server_ip

First Connection: Host Key Verification

On first connection you will see a host authenticity prompt. Accepting stores the server host key in ~/.ssh/known_hosts.

Typical prompt:

• The fingerprint shown is the server identity.
• Confirm it through a trusted channel if you manage the server (console, provider panel, prior record).

Verify server host key fingerprint (run on server)

sudo ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

If the server uses RSA host keys:

sudo ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

SSH Config Alias (Recommended)

A config alias makes connections consistent and avoids copy/paste errors.

Edit:

nano ~/.ssh/config

Example:

Host myvps
  HostName 123.123.123.123
  User user
  Port 2581
  IdentityFile ~/.ssh/mykey
  IdentitiesOnly yes

Use:

ssh myvps

Set secure permissions:

chmod 600 ~/.ssh/config

Advanced SSH config options (optional)

Keep connections alive (reduce idle disconnects):

Host myvps
  ServerAliveInterval 30
  ServerAliveCountMax 3

Separate known hosts for a specific environment:

Host myvps
  UserKnownHostsFile ~/.ssh/known_hosts_myvps

SSH Agent and Passphrases

Using a passphrase on the private key is recommended. To avoid typing it repeatedly, use ssh-agent.

Start agent (Linux/macOS, common shells)

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/mykey

List loaded keys:

ssh-add -l

Remove a key from agent:

ssh-add -d ~/.ssh/mykey

Remove all keys:

ssh-add -D

Windows Clients

MobaXterm

Steps:

1. Open MobaXterm

2. Session → SSH

3. Fill:

   • Remote host: server IP/hostname
   • Port: SSH port (example: 2581)
   • Username: remote user

4. Advanced SSH settings → enable Use private key

5. Select your private key file

Notes:

• MobaXterm often supports OpenSSH keys directly.
• If the key is not accepted, convert using MobaKeyGen.

PuTTY

PuTTY commonly uses .ppk keys.

1. Open PuTTYgen

2. Load an existing private key

3. Save as .ppk

4. Open PuTTY

   • Session: Hostname and port
   • Connection → SSH → Auth: select .ppk

5. Save session and connect

Troubleshooting (Detailed)

1) Permission denied (publickey)

Most common causes:

• Using the wrong key
• Wrong remote username
• authorized_keys missing or incorrect
• Bad permissions or ownership

Checks:

• Confirm the user is correct:

ssh -i ~/.ssh/mykey -p 2581 user@server_ip

• Force OpenSSH to use only the specified key:

ssh -o IdentitiesOnly=yes -i ~/.ssh/mykey -p 2581 user@server_ip

• Verify remote permissions (on server):

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

• Check remote logs (Ubuntu/Debian):

sudo tail -n 200 /var/log/auth.log

2) Server refused our key

Likely the public key is not correctly installed. Confirm authorized_keys contains a full, unwrapped public key line.

On server:

wc -l ~/.ssh/authorized_keys
tail -n 5 ~/.ssh/authorized_keys

3) Timeout / cannot reach SSH port

Possible causes:

• Wrong port
• Firewall blocks the port
• Cloud security group blocks the port
• SSH daemon not listening

On server, confirm sshd is listening:

sudo ss -tulpn | grep ssh

Confirm UFW:

sudo ufw status verbose

From local machine, check port reachability:

nc -vz server_ip 2581

4) Host key mismatch warning

If you see a warning about host identification changed, verify first. If legitimate (server rebuilt), remove only the specific entry:

ssh-keygen -R server_ip

Then reconnect and accept the new fingerprint after verification.

5) Too many authentication failures

If your client tries many keys, the server may disconnect before the correct key is used.

Fix by forcing a single key:

ssh -o IdentitiesOnly=yes -i ~/.ssh/mykey -p 2581 user@server_ip

Security Hardening (Recommended Pattern)

Disable password authentication (after key access is confirmed)

/etc/ssh/sshd_config:

PasswordAuthentication no

Restart:

sudo systemctl restart ssh

Disable root login

/etc/ssh/sshd_config:

PermitRootLogin no

Restart:

sudo systemctl restart ssh

Restrict which users can SSH

/etc/ssh/sshd_config:

AllowUsers user1 user2

Restart:

sudo systemctl restart ssh

Restrict SSH access by IP (UFW)

Allow from a trusted IP and deny all other SSH attempts:

sudo ufw allow from YOUR_PUBLIC_IP to any port 2581 proto tcp comment "SSH allow trusted IP"
sudo ufw deny 2581/tcp comment "SSH deny others"

Ensure correct rule order if needed:

sudo ufw status numbered
sudo ufw insert 1 allow from YOUR_PUBLIC_IP to any port 2581 proto tcp

Quick Checklist

Step	Action	Command
:	-
1	Confirm private key exists locally	ls -l ~/.ssh/mykey
2	Ensure local permissions are strict	chmod 700 ~/.ssh && chmod 600 ~/.ssh/mykey
3	Confirm server has the public key	cat ~/.ssh/authorized_keys
4	Ensure server permissions are strict	chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys
5	Connect using explicit key and port	ssh -o IdentitiesOnly=yes -i ~/.ssh/mykey -p 2581 user@server_ip
6	Store an alias for consistent access	~/.ssh/config
7	Validate sshd_config before restart	sudo sshd -t
8	Apply hardening after confirmation	disable passwords, restrict users, restrict by IP